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DEFENSE  SCIENCE 
BOARD 

Dr.  Paul  Kaminski 
Chairman,  Defense  Science  Board 

Office  of  the  Undersecretary  for  Acquisition  and  Technology 
The  Pentagon 
Washington,  D.C. 


Dear  Dr.  Kaminski: 

Enclosed  is  the  final  report  of  the  Defense  Science  Board  Task  Force  on  Acquiring  Defense 
Software  Commercially.  We  were  tasked  to  determine  the  conditions  under  which  procurement  of 
defense  software  can  use  commercial  practices  and  to  define  needed  changes  to  permit  such  use. 
In  the  attached  report,  we  make  specific  recommendations  with  regard  to  DoD  process  credibility, 
software  program  management,,  required  expertise  of  DOD  personnel  using  modern  software 
practices,  use  and  integration  of  commercial  off-the-shelf  software,  DoD  software  acquisition 
practices,  use  of  software  architectures  by  DoD  as  a  management  tool,  and  DoD’s  investment  in  the 
software  technology  base. 

Based  on  our  review,  we  concluded  that  issues  associated  with  defense  software  are 
applicable  across  the  wide  spectrum  of  DoD  software  intensive  systems.  However,  to  reap 
maximum  benefit  from  improvements  related  to  the  myriad  aspects  associated  with  software,  the 
DoD  requires  a  more  coordinated  approach  to  the  oversight  of  its  diverse  software  capabilities  and 
programs. 

To  provide  such  Department -wide  oversight  and  to  facilitate  implementation  of  the  other 
specific  recommendations  contained  within  this  report,  the  Task  Force  recommends  that  the 
Secretary  of  Defense  assign  to  the  Under  Secretary  of  Defense  (Acquisition  and  Technology)  the 
responsibility  for  DoD-wide  software  technology,  policy,  practices  and  acquisition.  This 
centralized  approach  will  best  serve  the  Department  for  all  software  intensive  systems  and 
programs.  To  support  the  implementation  of  this  recommendation  and  to  ensure  that  the  key 
stakeholders  are  participants  in  the  process,  the  Task  Force  further  recommends  the  formation  of 
an  Executive  Council  consisting  of  the  appropriate  principals  from  OSD,  the  Services,  and  the 
Defense  Agencies. 

We  thank  all  of  the  members  and  government  advisors  of  this  Task  Force  for  their 
dedicated  efforts  and  significant  contributions  to  this  study. 
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Executive  Summary 


<  The  Defense  Science  Board  Task  Force  on  Acquiring  Defense  Software  Commercially  recognizes 

that  DoD  systems  are  becoming  increasingly  dependent  on  the  use  of  software  as  the  mechanism  for 
implementing  operational  capabilities.  To  adapt  to  changing  military  and  national  security  situations,  DoD 
is  more  dependent  than  ever  on  its  ability  to  modify  mission  software  rapidly,  often  in  near  real-time. 
However,  software  remains  the  schedule  and  cost  driver  for  the  development  and  maintenance  of  many 
i  important  defense  systems. 

In  its  review  of  current  DoD  and  commercial  software  acquisition  practices,  the  Task  Force  found 
notable  differences,  as  evidenced  in  Appendix  C.  There  are,  however,  many  similarities  between  the 
various  categories  of  DoD  and  commercial  software  systems.  Although  there  are  indications  that 
commercial  development  efforts  have  achieved  better  predictability  and  lower  costs,  the  Task  Force  noted  a 
significant  lack  of  credible,  quantitative  data  to  substantiate  this  assessment. 

In  general,  the  Task  Force  concluded  that  DoD's  investment  in  software  requires  greater  DoD-wide 
management  control  and  oversight  in  the  coming  years  if  the  Department  is  to  exploit  the  use  of 
commercial  software  acquisition  practices  fully,  as  well  as  rapid  advances  in  software  technology.  The 
following  is  a  summary  of  selected  findings  and  recommendations  toward  that  end. 

Process  Credibility:  Current  DoD  practice  is  not  compatible  with  commercial  business 
practices.  DoD  should  work  to  make  necessary  changes  to  acquisition  regulations  such  as: 

•  Having  program  managers  manage  3  of  3  (price/schedule/functionality)  but  only  constrain  2  of  3 

•  Defining  successful  performance  on  contracts  as  delivering  a  solution  (with  predictable  price, 
schedule  and  functionality)  not  adherence  to  government  processes,  procedures  and  specifications 

•  Not  requiring  c-level  specifications  for  software  projects  developed  in  Ada 

•  Establishing  mechanisms  to  allow  both  current  ability  to  perform  as  well  as  past  performance  as 
key  factors  in  source  selection 

•  Encouraging  offerors  to  demonstrate  as  much  functionality  as  possible  as  part  of  bid  without 
eliminating  domain  knowledgeable  competition 

DoD  Program  Management:  DoD  program  management  approaches  discourage  the  use  of 
commercial  practices.  Program  managers  lack  incentives  to  tailor  procedures  to  fit  individual  program 
needs  or  to  develop  “corporate”  solutions  (e.g.,  employ  common  architecture  or  common  software 
components).  DoD  should  establish  and  implement  overarching  s  >ftware  life  cycle  guidelines  more 
conducive  to  the  use  of  commercial  practices  and  products,  such  as: 

•  Defining  software  architectures  to  enable  rapid  changes  and  reuse 

•  Facilitating  early  system  engineering  and  iterative  development 

•  Participating  in  development  of  commercial  and  international  standards 

•  Allowing  the  fielding  of  software  directly  from  test  beds  with  user  consent 

•  Requiring  program  managers  to  stay  with  programs  at  least  through  beta  testing  to  maintain 
continuity  of  understanding  of  original  nuances  in  requirements 


DOD  Personnel  There  is  currently  a  shortage  of  sufficiently  qualified  software  personnel  at  all 
levels  within  the  Department.  DoD  should  establish  a  Department-wide  software  program  management 
education  and  training  initiative  that  includes:  changing  courses  for  PMs  to  reflect  best  commercial 
practices  and  other  recommendations  of  this  task  force  and  providing  for  changes  to  reflect  the  dynamics 
of  the  software  industry;  rotating  government  and  contractor  personnel  between  PM  and  developer 
organizations  to  build  understanding  and  trust;  encouraging  use  of  IPA's  from  industry;  and  integrating 
software-qualified  personnel  into  senior  DoD  acquisition  staff. 

Use  and  Integration  of  Commercial  Off-the-Shelf  (COTS)  Software:  DoD  has  not 
fully  identified  the  pros  and  cons  associated  with  the  use  of  COTS  software  and,  as  a  result,  has  not 
determined  when  and  how  best  to  use  COTS  software.  To  facilitate  this  process,  DoD  should  require 
trade  studies  and  analysis  of  the  use  of  COTS  software  in  DoD’s  software  acquisition  process  where 
appropriate.  Further,  DoD  should  establish  “customer  friendly,"  application-specific  information 
technology  “component  stores”  to  enable  program  managers  to  assemble  systems  rather  than  develop  them 
through  use  of  reusable,  prequalified  components.  DoD  should  also  increase  technology  base  funding  for 
security  audit  tools  for  systems  employing  COTS  software  and  should  capitalize  on  innovative  cost- 
effective  techniques  for  acquiring  and  using  COTS  software  products,  such  as  the  use  of  enterprise 
licenses. 

Software  Architecture:  Software  architecture  was  emphr  Ized  by  the  Task  Force  as  a  means 
for  achieving  important  ends.  There  is  currently  little  emphasis  on  architecture  in  DoD  software  programs 
or  regulations.  As  a  result,  DoD  is  not  benefiting  from  architecture  as  a  key  tool  for  evolutionary 
development  and  for  early  (and  frequent)  involvement  of  users  with  functional  capability  and  facilitating 
reuse,  requirements  changes  with  minimum  cost  and  schedule,  and  “product  line”  management.  DoD 
should  require  vendors  to  propose,  manage  and  control  the  architecture  and  should  establish  an  early 
architecture  deliverable  in  all  developments. 

Software  Technology  Base:  The  current  DoD  software  technology  base  investment  does  not 
adequately  take  advantage  of  commercial  R&D.  Further,  software  technology  transfer  (both  internal  and 
external)  is  not  receiving  adequate  emphasis  within  DoD.  DoD  should  provide  for  the  evolution  of  the 
DoD  Software  Technology  Strategy  to  align  with  emerging  commercial  technology  and  practices. 

Overarching  Recommendations:  To  facilitate  implementation  of  the  many  recommendations 
contained  in  this  report,  the  Task  Force  concluded  that  DoD's  investments  in  software  require  greater 
management  control  and  DoD-wide  oversight.  To  this  end,  the  Task  Force  recommends  that  the  Secretary 
of  Defense  (SECDEF)  assign  the  Under  Secretary  of  Defense  (Acquisition  and  Technology)  (USD  (A&T)) 
the  responsibility  for  DoD-wide  software  technology,  policy,  practices  and  acquisition.  In  carrying  out 
this  responsibility,  the  Task  Force  recommends  that  the  USD  (A&T)  consider  forming  an  executive 
council  with  the  Director,  Defense  Research  and  Engineering  (DDR&E),  the  Assistant  Secretary  of 
Defense  (Command,  Control,  Communications  and  Intelligence)  (ASD  (C3I))  and  appropriate 
representatives  from  the  Services  and  Defense  Agencies  as  members.  Further,  USD  (A&T)  should 
provide  for  a  supporting  “process  action  team”  to  assist  in  implementation  of  the  recommendations  of  this 
Task  Force  study. 
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1.0  TASK  FORCE  OVERVIEW 
1 . 1  TERMS  OF  REFERENCE 


Terms  of  Reference 


Objectives: 

•  Determine: 

-  Conditions  Under  which  Procurement  of  Defense  Software  Can  Use  Commercial  Practices 

-  Changes  Required  to  Permit  Such  Use 

•  Develop  Strategy  that  Incorporates  Such  Practices 

-  Not  Constrained  by  Existing  DoD  Standards 

-  Viewed  as  Coexisting  Alternative  Strategy 

-  Includes  DoD  Use  of  Commercial  Software  Products 

•  Compare  Proposed  Strategy  with  Current  DoD  Strategy,  Indicating 
Circumstances  Where  Each  is  Most  Beneficial 

Scope: 

«  All  Software  Intensive  Systems 

•  All  Stages  of  the  Software  Life-Cycle 


Appendix  A  provides  the  Terms  of  Reference  by  which  the  Task  Force  was  established.  The 
objectives  and  scope  of  this  Task  Force  are  outlined  above.  At  the  first  meeting  of  the  Task  Force,  the 
sponsor  of  the  study  (the  Director,  Defense  Research  and  Engineering)  requested  that  the  Task  Force 
provide  a  strategy  that:  was  sensible,  pragmatic,  and  unconstrained  by  current  DoD  acquisition  practices; 
was  based  on  evaluation  of  mechanisms  for  integrating  defense  software  efforts  with  commercial  software 
efforts,  did  not  require  legislative  relief;  and  addressed  the  full  spectrum  of  DoD  software  applications. 
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1.2  CAVEATS 


Caveats 


•  Relied  on  Inputs  from  DoD  and  Industry  Experts 

•  Provided  No  Detailed,  Quantitative  Assessment  or 
Evaluation  of  Individual  Topics 


•  Did  Not  Address: 

-  Success  or  Failure  of  Ada 

-  Recommended  Software  Development  Approach  for  Specific 
Programs 

-  Specific  COTS  Products  for  DoD  to  Exploit 

-  Trade-Offs  Between  Hardware  and  Software 

_ 


The  Task  Force  relied  heavily  on  inputs  from  a  variety  of  DoD  and  industry  experts  regarding  a 
wide  range  of  topics  related  to  defense  software  technology,  policy,  practices  and  acquisition.  Appendix 
B  lists  the  many  briefings  that  were  provided.  Although  the  Task  Force  chose  not  to  provide  detailed, 
quantitative  assessments  or  evaluation  of  individual  topics,  it  used  the  information  associated  with  these 
topics  in  the  formulation  of  findings  and  recommendations.  The  areas  not  addressed  by  the  Task  Force, 
while  important  in  and  of  themselves,  were  determined  not  to  be  directly  germane  to  the  development  of  an 
overall  defense  software  strategy. 
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In  its  efforts  to  assess  the  appropriateness  of  DoD  use  of  commercial  practices,  the  Task  Force 
addressed  software  management,  contracting,  and  technical  issues.  The  above  viewgraph  lists  the  primary 
topics  considered  within  each  of  these  categories. 
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1.4  MEMBERSHIP 


Members 


Co-Chairmen: 

•  Dr.  George  H.  Heilmeier 

•  Dr.  Larry  Druffel 
Members: 
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•  Mr.  Jack  Hancock 

•  Mr.  Patrick  Hillier 

•  Mr.  Arthur  E.  Johnson 

•  Dr.  Bruce  Johnson 

o  Mr.  Aian  McLaughlin 

•  Dr.  Alvin  E.  Nashman 

•  Mr.  John  Stcnbit 

•  Dr.  Terry  Straeter 

Independent  DSB  Reviewers 

•  Mrs.  Joan  Habermann 

•  Mr.  Philip  A.  Odeen 


Bellcore 

Software  Engineering  Institute 


TASC 

Retired  Exec  VP,  Pacific  Bell 
EDS 

Loral  Federal  Systems 
Andersen  Consulting 
MIT  Lincoln  Labo-atory 
Computer  Sciences  Corporation 
TRW 

GDE  Systems,  Inc. 


Logistics  Management  Institute 
BDM  International,  Inc. 


Executive  Secretary 

•  Ms-  Virginia  L.  Castor 

DSB  Secretariat  Representative 

•  CDR  Robert  C.  Hardee 


ODDR&E/AT 


Go  vernment  Advisors 
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Ms.  Deborah  Castleman 

OASD(C3I) 

Mr.  Frank  Kendall 

OUSD(A&T) 

Mr.  Gene  Porter 

OUSD(A&T) 

Mr.  William  Mounts 

ODUSD(AR) 

Ms.  Linda  Brown 

ODASD(IM) 

Mr-  Chris  Dipetto 

OUSD(A&T) 

Joint  Staff: 

• 

Mr.  Joseph  Toma 

JSA 

Services: 

• 

LTG  Peter  Kind 

Army 

<& 

RADM  John  G.  Hekman 

Navy 

« 

Mr.  Lloyd  K,  Mosemann  II 

Air  Force 

zencies: 

• 

Ms.  Belkis  Leong-Hong 

DISA 

• 

Dr.  Ed  Thompson 

ARFA 

Task  Force  members  represented  a  valued  cross  section  of  software  expertise  within  both  the  DoD 
and  commercial  sectors.  The  government  advisors  represented  senior  executives  (including  the  three 
Service  Software  Executive  Officials)  from  the  major  software  management  organizations  within  the 
Department. 
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2.0  CURRENT  POD  AND  COMMERCIAL  SOFTWARE  ACQUISITION  PRACTICES 


2.1  BACKGROUND 


Background:  Previous  Studies 


DSB  Summer  Study  on  Technology  Base  (1981) 

Joint  Service  Task  Force  on  Software  Problems  (1982) 

AF  SAB  High  Cost  and  Risk  of  Mission  Critical  Software  (1983) 

CODSIA  Report  on  DoD  Management  of  Mission-Critical  Computer  Resources  (1984) 
DSB  Task  Force  on  Military  Software  (1987) 

Ada  Board  Response  to  DSB  Task  Force  (1988) 

Summer  Report  on  Defense-Wide  Audit  of  Support  for  Tactical  Software  (1988) 
Workshop  on  Executive  Software  Issues  (1988) 

Workshop  on  Military  Software  (1988) 

Army  Materiel  Command  Study  (1989) 

Software  Technology  Development  and  Deployment  Plan  for  DoD  Technology  Base 
(1989) 

AF  SAB  Adapting  Software  Development  Policies  to  Modem  Technology  (1989) 

Draft  DoD  Software  Master  Plan  (1990) 

Draft  DoD  Software  Technology  Strategy  (1991) 

AF  SAB  Study  on  Information  Architecture  (1993) 

Study  on  Military  Standards  Impacts  on  the  Acquisition  Process  (1993) 

Draft  Software  Action  Plan  Working  Group  Report  (1993) 

Evolutionary  Acquisition  Study,  AFCEA  (June  1993) 


As  a  point  of  departure,  the  Task  Force  noted  a  number  of  previous  studies  addressing  issues 
related  to  the  defense  software  technology,  policy,  practices  and  acquisition.  Despite  the  increased 
emphasis  given  to  software  issues  by  the  DoD  (as  evidenced  by  the  above  list),  the  majority  of  the 
recommendations  resulting  from  these  studies  have  not  been  implemented. 
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Background:  Impediments  to  Change 


•  DoD  Software  Management 

No  Single  Operative  Mechanism  Exists  to  Implement  Change 
Three  Separate  Domains:  MIS,  C3I,  Embedded 
Two  Separate  Acquisition  Management  Structures 
Bureaucratic  "Turf"  Inhibits  Real  Reform 

•  Acquisition  Process 

5000  and  8000-Series  Developments  Typically  Employ  "Waterfall" 

Approach;  Not  Incremental/Spiral  Approach 

•  Culture 

Systems  Are  Stove  Pipe  --  My  System,  My  Program  (PM  is  King) 

DoD  Acquisition  Training  Reinforces  the  Wrong  Approaches 

•  Procurement 

The  Contracting  Process  Inhibits  Creativity  and  Investment  by 

Contractors;  Limits  Options 

Interpretation  of  Competition  in  Contracting  Act 

V _  J 

To  ensure  that  its  recommendations  could  be  readily  implemented  by  the  DoD,  the  Task  Force 
identified  the  primary  reasons  why  recommendations  from  previous  studies  had  not  been  acted  upon.  The 
Task  Force  then  formulated  its  recommendations  to  appropriately  address  these  impediments. 
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Why  Is  This  Study  Important? 


•  DoD  Military  Capability  Increasingly  Software  Dependent 

w  Software  One  of  Few  DoD  Budget  Areas  That  is  Projected  to  Grow 

-  Software  Has  Become  the  Overall  Defense  System  Schedule  and  Cost  Driver 

»  Declining  DoD  Budget  —  Affordability  a  Major  Concern 
»  Escalating  Costs  of  Software  Development  and  Life  Cycle 

-  Significant  Level  of  DoD  Resources  To  Be  Hpent  on  Information  Technology: 

»  FY92-FY94:  -$10  Billion/Year* 

»  EIA  Forecast  for  FY95  -  FY98:  ~$10  Billion/Year 
»  In-House  vs.  Contracted  Out;  30%  In;  70%  Out 

-  Will  Require  Greater  Management  Control 

•  Rich  Commercial  Base  to  Tap;  Many  Opportunities 

-  Custom  Software  -  Acquisition  Methodologies 

-  Off-the-Shelf  Products  -  Approach  to  Requirements  Determination 

•  Functionality  and  Flexibility  More  Embedded  in  Software  than  Hardware 

•  Study  is  Timely  Because  DoD  Leadership  is  Focused  on  Acquisition 
Reform 

-  Something  May  Actually  Get  Done 


Given  its  increasing  reliance  upon  software  as  the  mechanism  for  implementing  system 
capabilities,  coupled  with  the  rising  costs  associated  with  software  development  and  maintenance,  DoD 
must  take  action  now  to  address  the  issues  associated  with  software  acquisition.  The  commercial  sector 
provides  numerous  opportunities  upon  which  the  DoD  could  readily  capitalize.  The  time  is  ripe  for 
assertive  DoD  action,  particularly  since  the  current  leadership  is  so  strongly  focused  on  acquisition  reform. 
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The  Software  Domain 
is  Very  Large 


CUSTOM 
COMMERCIAL 
REAL  TIME  SYSTEMS 


LARGE  DOD  EMBEDDED  REAL 
TIME  SYSTEMS 


COMPLEXITY 


COMMERCIAL 

ENGINEERING 

SYSTEMS 


SMALL  DOD 
EMBEDDED  REAL 
TIME  SYSTEMS 


DOD 

ENGINEERING 

SYSTEMS 


CUSTOM  DOD 
BUSINESS  SYSTEMS 


CUSTOM  COMMERCIAL 
BUSINESS  SYSTEMS 


COMPLEX 

COMMERCIAL 

PRODUCTS 


=TT 

;ms  J 


/ - L 

COTS 

RELEASES 

V 

SMALL 

COMMERCIAL 
COTS  PRODUCTS 


COST  TO  DEVELOP 


The  software  domain  reviewed  by  the  Task  Force  encompasses  a  wide  variety  of  DoD  systems, 
ranging  from  software  tools  to  large  embedded  real-time,  systems.  The  associated  cost,  complexity,  and 
time  required  to  develop  these  systems  van,'  widely,  both  for  commercial  and  military  applications. 
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Background: 

DoD  Software  Acquisition  Management 


•  Two  Different  Sets  of  Software  Policies/Rules  Managed  by 
Two  Different  Organizations 

-  USD  (A&T)  for  Software  Embedded  in  Weapons/Systems 

-  ASD  (C3I)  for  C3I  Software  and  MIS  Applications 


•  Majority  of  Software  Issues  are  Applicable  to  All  DoD 
Systems 

•  Growing  Similarity  Between  DoD  and  Commercial  Software 
Developments  Across  the  Various  Types  of  DoD  Software 

-  Modem  Software  Tending  to  Blur  Distinction  Between  DoD  and 
Commercial  Applications 


Central  DoD  Leadership  Needed  More  than  Ever 

-  Major  Revisions  Needed  in  DoD  Software  Policies/Rules  and 
Management  Across  All  DoD  Applications  in  Order  to  Meet  SECDEF's 
Acquisition  Reform  Goals 

-  Interpretation  and  Implementation  of  DoD-Wide  Policies/Rules  by 
Management  is  a  Central  Issue 


Today,  the  management  of  DoD  software  acquisitions  is  quite  complex.  There  are  two  sets  of 
software  policies/rules  managed  by  two  different  organizations 

•  USD  (A&T)  for  software  embedded  in  weapons/systems 

•  ASD  (C3I)  for  C3I  software  and  MIS  applications 


There  currently  exists  within  the  DoD  a  dichotomous  organizational  structure  for  the  management 
of  DoD  software  intensive  systems.  The  Under  Secretary  of  Defense  (Acquisition  and  Technology)  is 
responsible  for  weapons  systems;  the  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications,  and  Intelligence  (C3I))  is  responsible  for  C3I  systems  and  Management  Information 
Systems  (MIS).  However,  as  technology  has  evolved  over  the  years,  the  issues  associated  with  the  use  of 

CA^fll/CU*A  Itt  all  rtf  thaca  oirotorvir  lioi/ft  nnmn  f  I  ’ti  /-»  r.  —  ,  ..f. 

ounVVuiv  aa*  On  uivov  ojotwiiia  uuvv  uwvuuiv  i/aai/uuauy  uav  dtuiiw.  i  nc  lObuaniA;  \ji  uiucicui  MJllWaiC- 

related  policies  and  regulations  that  are  oriented  according  to  the  type  of  system  is  no  longer  appropriate. 
If  the  DoD  is  to  adequately  address  this  fundamental  issue,  central  focused  management  is  required. 
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2.2  COMPARISON  OF  DEFENSE  AND  COMMERCIAL  SOFTWARE 


Comparison  of 

Defense  and  Commercial  Software  Needs 


•  Defense  and  Commercial  Software  Applications  Needs  Are  Merging 


Breadth  of 
Application 

IHESSEESSH 

■KSlEShH 

■jRBi 

■SH 

Sensitivity  to 
Enure 

Unique 

400,000 

Complex; 

Inflexible 

High 

-  DoD  Real  Time  Software 

-  CoDunrciAlTluctom  Software  Inlegrated 
Into  Real-Time  Operations 

Unique 

400,000 

IlllUfii 

Moving  Toward 
Wide 

200,000 

Complex;  Flexible 

Moderate 

-  C3I 

-  Commercial  tusiom  Software  Integrated 
Into  Business  Operations 

Moving  1  oward 
Wide 

200,000 

ESSEHFm 

Moderate 

MIS  Systems 

-  DoD  Automated  Information  Systems 

Very  Wide 

200,000 

Moving  Toward 
Commercial 

Low 

-  Commercial  Automated  Information 
Systems 

Very  Wide 

200,000 

Exploits  Object 
Oriented 
Technology 

Low 

VAty  Wide 

50,000 

Tied  to  Weapcn 
System  Cycle 

Low  to  High 
(Depending  on  Use) 

-  DoD  Component  Stores 

-  Commercial  Shrink  Wrapped 

Very  Wide 

50,000 

Tied  to 

Commercial  Cycle 

Lew 

DoD  and  commercial  applications  for  software  are  different  in  some  ways  but  very  similar  in 
others.  The  above  table  highlights  these  similarities  and  differences  for  real-time  applications,  command 
and  control  applications,  MIS  systems  and  reusable  components  and  products.  This  apparent  disparity  in 
the  classification  of  software  between  DoD  and  commercial  vendors  increases  the  complexity  of  DoD's 
management  task.  Companies  (people,  methods,  and  organizations)  are  usually  specialized  to  one  or  more 
commercial  type  systems,  not  to  DoD  type  systems.  Defense  and  commercial  software  applications  needs 
are  merging,  providing  the  potential  that  DoD  can  exploit  commercial  capabilities  more  effectively  over 
time.  Major  revisions  are  needed  in  DoD  software  management  across  all  DoD  applications  in  order  for 
DoD  to  capitalize  on  the  evolving  commercial  base. 
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Process  Comparison  Summaiy 


Civilian I 

Custom  Custom  Contmarciai  Commercial 

Process  Attribute  DoD  Commercial  Product  Resort 


Problem  Definition 

Own**  by  CMgkulM 

Shared  (Irtcludee 
Martlet  Anafyala) 

bb 

Mote  Practical 
Requirement* 

Proceee  Definition 

By  Spw,  R(0td 

Inftemet/f  solving 

ln**fn*MEvoMi*8 

Reduced  Coat/ 
Schedule 

Flexibility 

VwyUmittd 

Not  Coru  trained 

&***•* y 
iMbmtted 

Proceee  Improvement 
EaM.  Enabling 

Technology* 

MUtetonee/R*  views 
(Duration.  Formality) 

Unearned, 
Haevy  Document 

Short,  MmmI 

Reduced  Coat/ 
Schedule 

duetomer/Uier 

Involvement  In 

Development 

BMP 

Leea  Hequlrements 
Churning.  Reduced 
Coet/Schedule 

Process  Monitoring  by 
Cuntonter 

HMtvy 

SOflM 

kt- _ 

nans 

Reduced  Coat/ 
Schedule 

Customer  Acceptance 
Prooeea  * 

.  Wgorou*  ■;  - 

Simp** 

.  MarMPMO* 

Reduced  Coni/ 
Schedule 

InapecUon/Teeting 

nicorouc /Fonul 

Rigor  ouaVorrnal 

Mpomui/fooRPl  . 

Similar  Product 
Quality 

Subcontracting 

BySf»c 

BrM  Product  Sp«. 
ISO  PoulfcM 

Brief  Product  fcpec 
BOMbk 

Quelity/Depandsjillty 
of  Subconiraclor  May 

a*  leae 

Uw  ol  XdvanoecT 
Development  Technique* 

(l.e.  Reue*/4CUOD) 

ImpUrirty 

OtocMiraeMi 

Extensive 

EltMMi** 

boduced  C«U 

Sc  had  lie  Higher 

Quality 

Source:  IBM  Federal  Systems 


ua 


As  is  evident  from  the  process  comparison  summary  above,  there  are  not  only  differences  between 
commercial  and  defense  applications,  but  also  in  the  process  used  by  each  to  develop,  acquire  and  support 
complex  software  systems.  Defense  and  other  federal  acquisition  regulations  and  detailed  specifications 
require  a  much  more  complex  set  of  deliverables  within  the  context  of  DoD’s  contracting  process. 
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Comparison  ofDoD  and  Commercial  > 
Software  Acquisition  Practices 


9  Requirements  Definition 

-  Commercial 

-  More  flexible  and  open  between  users  and  supplier 

-  Based  oil  strategic  plan  *nd  usually  cost/schedule,  driven 

-  More  willing  to  adjust  requirements  based  on  availability  of  off-the-shelf 
products 

-  Evolves  capability 

9  Vendor  Selection 

-  Commercial 

-  Much  more  flexible;  no  requirement  for  fairness  or  tc  maintain  the  public  trust 

-  Encourages  vendors  to  offer  best  solution,,  not  meet  100%  of  requirements 

-  Accommodate  teaming  and  long-term  relationships 

9  Development  Process 

-  Commercial 

-  More  flexible;  product  improvements  anticipated 

-  Team  approach  with  bias  toward  reuse  and  tailoring  of  existing  systems 

-  Multi-year  acquisitions  not  re-justified  each  year. 


In  essence,  the  Task  Force  found  major  differences  between  DoD  and  commercial  software 
acquisition  practices,  as  outlined  above  and  on  the  next  page. 


Comparison  ofDoD  and  Commercial 
Software  Acquisition  Practices  ( Cont ) 


a  Business  Practices 

-  Commercial  practice  more  flexible  with  greater  incentives 


•  Integration  Testing  and  Delivery 

-  Commercial  provides  integration  and  functional  testing  according  to  need 

-  DoD  uses  separate  test  agency  with  added  time  and  complexity 

-  Absence  of  beta  testing  within  DOD  increases  costs 


•  Rights  in  Data 

-  Commercial  more  flexible,  especially  regarding  resales 


•  Maintenance 

-  Commercial:  Maintenance  considered  and  integrated  with  development 

-  DoD:  Maintenance  not  major  factor  in  development  process 


Appendix  C  provides  a  more  complete  comparison  of  DoD  and  commercial  software  acquisition 
practices  as  developed  by  this  Task  Force. 


i 
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Assessment  of  Current  DoD 
Software  Acquisition  Approach 


•  Strengths 

-  Highly  Structured  Process  Tied  to  Individual  System 
Developments 

-  Tightly  Defined  Requirements 

-  Produces  High  Quality  Product  for  Mission  Critical  Systems 
That  Demand  Extremely  Low  Failure  Rates  (e.gv  Flight 
Controls  for  Man-Rated  Platforms) 


The  strengths  of  the  current  DoD  approach  to  software  development,  acquisition  and  operation  are 
summarized  above.  The  Task  Force  found  that  the  highly  structured  DoD  process  has,  in  fact,  provided  a 
high  quality  software  product,  in  most  cases. 
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Assessment  of  Current  DoD 
Software  Acquisition  Approach 


Weaknesses 

-  High  Life  Cycle  Cost  in  Time  and  Dollars 

-  Software  Development  Cycle  Tied  to  Weapon  System  Developments 

»  Incredibly  Long ,  Typically  13  to  15  Years  from  Concept  to  Fielding 

-  Encourages  Excessive  Acquisition  Agent  Involvement  in  Design 
Detail  and  Process 

-  Based  on  Mistrust  vs.  Mutual  Trust 

»  Excessive  Documentation 
»  Excessive  Formal  Review 
»  Excessive  Testing  of  Non-Criiicai  Systems 

»  Poor  Communication  Between  Vendor,  Acquisition  Agent  and  User 

-  No  Requirements  Addressing  Cost  and  Schedule 

-  Traditional  Approach  Used:  Design  it  All  and  Then  Build  it 

-  Little  or  No  Requirements  Relaxation  for  High  Cost  Items 

-  Inadequate  Beta  Testing  in  Early  Phase 

-  Little  Focus  on  Designing  in  Reusability 


However,  there  are  a  number  of  weaknesses  associated  with  the  current  defense  approach,  as 
summarized  above.  Many  of  these  weaknesses  derive  from  the  need  for  a  fair  and  open  procurement 
process  and  the  necessity  to  prove  that  public  dollars  are  wisely  spent. 
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Assessment  of  Current  Commercial 
Software  Acquisition  Approach 

Strengths 

-  Open  Architecture  Compatible  with  Usage  of  COTS  Software 

-  Much  Less  Formal  Competitive  Procurement  Process 

»  Includes  Prior  Performance  as  Major  Selection  Criterion 

-  Trend  Toward  Joint  Assumption  of  Risks  Between  Buyers  and  Suppliers 
Across  Development,  Operation,  Maintenance  and  Modernization 

-  Process  is  More  Flexible 

-  Shorter  Cycle  (Product  Release)  Times 

-  Tailorable  Level  of  Documentation  and  Oversight 

-  Emphasis  on  Reuse  and  Tailoring  Requirements  to  Existing  Products 

-  Beta  Testing  Widely  Used 

Weaknesses 

-  Less  Responsive  to  Continual  Changes  in  Requirements 

-  Less  Assurance  that  Software  Will  Function  Properly  Under  All  Situations 

-  Potentially  Locked  into  One  Vendor's  Proprietary  Application 


) 


The  strengths  and  weaknesses  of  the  current  commercial  approach  to  software  development, 
acquisition  and  operation  are  summarized  above. 
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Principal  Reasons  DoD  Software 
Programs  Get  Into  Trouble 


•  Poor  Requirements  Definition 

-  Lack  of  User  Involvement  in  Development  Process 

-  Inability  of  Users  to  Foresee  Benefits  of  Automation  Without  Incremental  Capability 

•  Inadequate  Software  Process  Management  and  Control  by  Contractor 

•  Lack  of  Integrated  Product  Teams 

-  Failure  to  Establish  "Team"  With  Vendors  and  Users 

-  Little  Participation  of  Functional  Area  Experts 

•  Ineffective  Subcontractor  Management 

•  Lack  of  Consistent  Attention  to  Software  Process 

•  Too  Little  Attention  to  Software  Architecture 

•  Poorly  Defined  and  Inadequately  Controlled  Interfaces  Between  Computer 
Hardware,  Communications  and  Software 

•  Assumption  That  Software  Upgrades  Can  "Fix"  Hardware  Deficiencies 
(Without  Assessment  of  Cost  and  Schedule  Risks) 

•  Focus  on  Innovation  Rather  than  Cost  and  Risk 

•  Limited  or  No  Tailoring  of  Military  Specifications  Based  on  Continuing 
Cost-Benefit  Evaluations 


V. 


Over  the  last  two  decades,  a  number  of  Do D  software  development  efforts  have  gotten  into  trouble, 
particularly  in  terms  of  actual  costs  and  schedule  vs.  expected  or  predicted  costs  and  schedule.  The  Task 
Force  heard  briefings  from  a  number  of  DoD  program  managers  where  this  was  the  case.  Based  on  the 
specific  programs  discussed  and  on  other  inputs,  the  Task  Force  prepared  a  listing  of  the  principal  reasons 
DoD  software  programs  have  gotten  into  trouble  as  shown  above.  Certain  of  these  reasons  are  a  reflection 
of  DoD  practice.  Others  are  tied  to  the  software  engineering  level  available  at  the  time  such  programs  were 
initiated.  The  Task  Force  believes  that  today’s  software  technology  and  practices  can  directly  address 
many  of  the  root  causes  for  such  past  problems. 
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Resource  Allocation: 
Where  Does  the  Effort  Go? 


Source:  Magnavox  and  Capers  Jones 


Military 

Commercial 

•  Engineering 

30% 

50% 

•  Evaluation 

20% 

20% 

•  Management 

15% 

10% 

•  Meeting  Support 

15% 

5% 

•  Documentation 

15% 

7% 

•  Customer/User  Support 

5% 

8% 

There  are  some  indications  that  commercial  development  efforts  have  achieved  better  predictability 
and  lower  costs  than  DoD  counterparts.  The  Task  Force  solicited  quantitative  data  on  this  issue  from  both 
government  and  commercial  software  experts.  The  figure  above  summarizes  one  type  of  indicator  of  the 
difference  between  commercial  and  government  projects  (in  terms  of  the  percentage  of  the  effort  expended 
for  different  aspects  of  a  typical  development). 


g 

0. 
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f  Comparison  of 

Commercial  and  Government  Projects 


— 

Application  Type 

— — 

Average  Size  (SLOC- 
New  and  Modified) 

■ 

flRSSiHi 

Commercial 

1491 

.•#9' jt 

26,000 

12.52 

V  t '+>  .C  -5^  .‘Cj 

W6wfc»Wwi»<wrt*A3>«jfeiF^ 

Government 

75 

26,000 

15.1 

Percent  Difference 

+  17.09% 

+  44.24% 

Commercial 

26,000 

gigglgglsg 

65.7 

Government 

29 

26,000 

20.9 

117.3 

Percent  Difference 

+  16.75% 

+  43.99% 

Commercial 

295 

212,000 

ESSS 

Government 

21 

212,000 

256 

242.9 

Percent  Difference 

+ 14.06% 

+  38.16% 

Commercial 

56 

442,000 

45 

1736 

Government* 

7 

442,000 

52 

2740 

Percent  Difference 

+  13.46% 

+  36.64% 

Summary  of 

Average  Percent  Difference 

+  15.34% 

+  40.76% 

*  Small  sample  may  be  statistically  invalid. 
Source:  QSM,  Inc. 


This  figures  summarizes  other  quantitative  indicators  of  the  difference  between  commercial  and 
government  projects  (in  terms  of  the  typical  size  of  the  code,  time  to  develop  the  application  and  overall 
level  of  effort). 

It  should  be  noted  that  the  Task  Force  was  unable  to  find  reliable,  quantitative  data  supporting  the 
notion  that  commercial  practices  are  more  cost-effective  than  DoD  piactices.  This  lack  of  reliable 
indicators  was  a  major  concern  of  the  Task  Force. 
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5.0  MAJOR  FINDINGS  AND  RECOMMENDATIONS 
3.1  PROCESS  CREDIBILITY 


Process  Credibility 


Findings 

•  Attempts  to  achieve  absolute  fairness  in  competition  in  contracting 
(fair  and  reasonable  pricing)  have  led  to  a  lack  of  trust  between 
government  and  individuals/contraciors 

-  Current  DoD  practice: 

»  Is  not  compatible  with  commercial  business  practices 
»  Is  focused  on  contractor  management/audit  activities  and  costs 
»  Hinders  flexibility  (managers  take  no  pe.-onal  risk) 

»  Is  not  well  suited  to  procuring  complex,  knowledge-intense,  “first  of  a  kind"  systems 
»  Is  too  costly 

Does  not  prevent  malfeasance  or  incon.petenee 
■>  Leads  to  adversarial  relationships 

»  Reduces  reuse  and  contractor  incentive  to  invest  because  of  stringent  data  rights 
interpretation 

-  Contractors  must  make  profit  on  single  contract;  no  long  term  relationship  (DoD 
business  practices  do  not  view  profit  as  a  legitimate  cost  of  doing  business) 

-  No  individual  fully  understands  or  owns  total  process 

V _ III _ J 


The  Task  Force  spent  considerable  effort  on  how  the  requirement  for  DoD  to  ensure  public  trust  in 
its  acquisition  process  influences  DoD’s  ability  to  employ  “commercial  best  practices.”  The  Task  Force 
found  that  attempts  to  achieve  absolute  fairness  in  competition  in  contracting  have,  in  fact,  led  to  a  lack  of 
trust  between  the  government  and  individuals/contractors.  DoD  acquisition  processes  are  focused  c  i 
contractor  audit  activities  to  an  extent  that  hinders  the  flexibility  of  Program  Managers  and  contracto  >. 
DoD  PMs  are  not  incentivized  to  assume  any  personal  risk  associated  with  allowing  for  flexibility 
comparable  to  commercial  practice.  Criminal  sanctions  are  a  significant  disincentive  for  such  efforts. 

DoD’s  acquisition  system  still  does  not  prevent  malfeasance  or  incompetence  and  leads  to 
adversarial  relationships  rather  than  partnerships  which  are  the  norm  within  commercial  industry  software 
developments.  One  problem  highlighted  by  the  Task  Force  was  that  the  current  DoD  system  does  not 
allow  one  individual  or  manager  to  conti  >1  the  total  process,  even  for  a  specific  project.  This  lack  of 
control  leads  to  a  diffusion  of  accountability  and  hinders  DoD’s  ability  to  oversee  complex  “first  of  a 
kind”  software  developments. 
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Process  Credibility 


Findings 

•  Requirements  and  Source  Selection  Inflexibility 

-  Vendors  attempt  to  meet  every  requirement 

-  Ease  of  protest 

-  Requirements  focus  vendor  on  particular  solution 

-  Requirements  have  became  alternative  to  professional  judgment 

-  Departures  from  requirements  have  caused  protests  and  bad  publicity 

•  Price/Schedule/Functionality 

-  In  commercial  sector,  managers  constrain  2  out  of  3  (e.g.,  cost  and  schedule) 

-  In  DoD,  managers  constrain  3  out  of  3 

•  Constrained  Communication  During  Solicitation 

-  Questions  are  provided  to  competitors  (can  give  away  proprietary  concepts) 

-  Questions  are  often  misinterpreted  and  answered  incorrectly 

-  Guarded  way  of  asking  questions  limits  substantive  feedback 

•  Complicated  Regulations 

-  Restrict  variety  of  proposals 

-  Restrict  competition  and  limit  government  options 

-  Consume  time  and  resources 

-  Drive  government  employees  to  follow  conservative  procedures  as  safest  path  (inhibits  "best  value") 

•  Government-Unique  Accounting  Procedures  and  Audits 

-  Audit  requirements  limit  available  vendors 

-  Add  substantial  cost 

-  Create  need  for  separate  accounting  systems 

-  Drive  vendor  to  lowest  labor  cost  solution  rather  than  best  value  solution 

-  Lead  to  rejection  of  good  solutions  based  on  value  pricing  vs  cost-based  pricing 


The  viewgraph  above  lists  important  hindrances  that  the  Task  Force  sees  with  regard  to  the 
adoption  of  commercial  software  practices.  Many  of  the  Task  Force  recommendations  address  these 
hindrances. 
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Process  Credibility 


Recommendations 


•  Make  necessary  changes  to  acquisition  regulations 


-  Have  Program  Managers  Manage  3  of  3  (Price/Schedule/Functionality)  But  Only 
Constrain  2  of  3 

-  Define  Successful  Performance  on  Contracts  as  Delivering  Solution  (with  Predictable 
Price,  Schedule  and  Functionality)  Not  Adherence  to  Government  Processes, 
Procedures  and  Specifications 

--  Do  Not  Require  C-Level  Specifications  for  Software  Projects  Developed  in  Ada 

-  Prior  to  RFP,  Government  Should  Perform  Independent  Market  Analyses  of  Off-the- 
Shelf  and  Contractor  Products  to  Assure  "Best  Value"  Solution 


Establish  Mechanisms  to  Allow  Both  Current  Ability  to  Perform  as  Well  as  Past 
Performance  as  Key  Factors  in  Source  Selection 

*>  Require  Source  Selection  Evaluation  of  Development  Contractor's  Through  a  Formal  Software 
Process  Capability  Evaluation 

Encourage  Offerors  to  Demonstrate  as  Much  Functionality  as  Possible  as  Part  of  Bid 
Without  Eliminating  Domain  Knowledgeable  Competition 
»  Executable  Architecture  as  a  Minimum 
»  Weight  Heavily  in  Selection 


The  Task  Force  makes  the  above  recommendations  with  regard  to  process  credibility. 


3.2  DOD  PROGRAM  MANAGEMENT 


DoD  Program  Management 


Findings 

•  Program  management  does  not  encourage  "80%  solution  for  20%  cost" 

•  Users  often  not  significantly  involved  in  process 

•  Little  emphasis  on  life-cycle  issues  (including  maintenance  and  support) 

•  Existing  policies,  methodologies,  and  procedures,  and  their  implementation  are 
inadequate 

-  Little  evidence  that  policies  are  influenced  by  actual  experience  and  vice  versa 

-  Little  effort  to  measure  effectiveness  and  costs  of  policy  directives 

•  Some  indication  that  DoD  is  migrating  toward  development  and  use  of 
standards-based  architectures 


Program  Managers  lack  incentives  to  allow  tailoring  of  procedures  to  fit 
individual  program  needs  or  to  develop  "corporate"  solution  (e.g.,  employ 
common  architecture  or  common  software  components) 


The  Task  Force  found  that  the  DoD  program  management  system  itself  discourages  the  use  of 
commercial  best  practices.  These  findings  are  not  unique  to  software,  but  are  particularly  important  for 
software  given  the  significant  cost  savings  associated  with  software  reuse.  There  are  few  incentives  for 
Program  Managers  (PMs)  to  develop  or  even  employ  corporate  solutions  (common  architectures/software 
components),  particularly  if  they  are  more  expensive  to  acquire.  Rather,  each  PM  will  tend  to  optimize 
his/her  development  for  its  own  purpose.  Further,  it  is  difficult  for  users  to  be  significantly  involved  until 
late  in  a  software  development  process,  unless  some  sort  of  prototype  can  be  constructed.  DoD  PMs  place 
little  emphasis  on  life-cycle  issues  (such  as  software  maintenance  and  support).  Existing  DoD-wide 
software  policies,  methodologies,  and  procedures,  and  their  implementation  by  PMs  are  inadequate. 
There  is  little  evidence  that  policies  are  influenced  by  actual  experience  and  vice  versa  and  there  is  little 
effort  to  measure  the  effectiveness  and  costs  of  policy  directives. 
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DoD  Program  Management  (cant.) 


Recommendations 

•  Establish  Overarching  Software  Life  Cycle  Guidelines 

-  Tools/Methods 

»  Define  Software  Architecture*  to  Enable  Rapid  Change*  and  Reuae 

i  To  Achieve  the  Benefit*  of  Using  Standarde-Baied  Architecture*,  DoD  Mult  Manage  Program*  liking: 

•  Early  »y»l*tr  engineering 

•  Iterative  development 

e  Proactive  participation  In  development  of  tHeee  aUndarda 

-  Promote  Dtvdlopmtnl/UM  of  Community-Wide  Metric*  and  Models  (*.g.,  SEI's  Capability  Maturity  Model) 

-  Acquisition 

>•  Revise  the  Milestones  for  Software-Intensive  Development 

•  Addreaa  the  need  for  a  eoftwarv-flrM  phileoophy 

•  Provide  for  a  layered  MflwiMAtdwiN  atandarda  baaed  architecture 

•  AMfulaitioin  and  life  cycle  planning  should  new  aeparale  hardware  and  wlhrare  fieldlnga  bated  on  the  buelneea  mam  ef  the  epecifU  prelect 

•  Require  Early  Interaction  Between  User*  Acquisition  Agent*  and  Developer;  Identify  and  Get  Early  User  Involvement 
»  Apply  Evolutionary  Development  with  Rapid  Deployment  of  Initial  Functional  Capability 
»  Encourage  Competition  of  Technical  Approach  vs.  Cost 

»  Provide  Incentives  and  Guidelines  to  Encourage  Software  Reuse  (Architecture-Based  Reuse) 

»  Reduce  Documentation  and  Review  Requirements  for  “Mature"  Companies  <i,ev  Companies  Determined  to  Be  “Mature" 
Through  Evaluation  Mechanisms) 

»  Tailor  operational  testing  to  develop  DoD  “Beta  Test"  philosophy 

•  A  Hew  fielding  ef  seftwire  direct  frore  tret  bed*  with  UMr  emuent 
»  Have  Program  Manager  Stay  with  Progvama  at  Laaat  Through  Beta  Testing  to  Maintain 

Original  Nuances  in  Requirements 


The  Task  Force  makes  the  recommendation  that  DoD  establish  overarching  software  lifecycle 
guidelines  directed  at  facilitating  program  manager  employment  of  commercial  practices  and  software  and 
that  a  DoD-wide  effort  be  made  to  oversee  implementation  of  these  guidelines.  The  tools,  methods  and 
acquisition  approaches  recommended  by  the  Task  Force  are  listed  above. 


3.3  DOD  PERSONNEL 


DoD  Personnel 


IM 


I 

I 


Findings 

•  A  shortage  of  sufficiently  qualified  software  personnel 
currently  exists  at  all  levels  within  the  DoD 

-  Expertise  for  software  acquisitions,  software  evaluations,  and  software 
maintenance/support 

-  Expertise  to  represent  DoD  (customer)  interests  with  commercial  sector 

-  Expertise  in  domain  software  design  and  applications  | 

-  Expertise  in  software  technology  to  develop  policies,  standards,  and  | 

guidelines 

-  Expertise  in  software  program  management 

V _ ) 


Based  on  the  inputs  provided,  the  Task  Force  found  that  a  shortage  of  sufficiently  qualified 
software  personnel  exists  at  all  levels  within  the  DoD.  Without  personnel  who  are  highly  qualified  in 
modern  software  practices,  DoD  will  not  be  as  capable  of  effectively  exploiting  complex  software  within 
its  systems.  This  personnel  shortage  has  been  a  major  contributor  to  the  problems  that  have  arisen  in  past 
DoD  software  development  programs. 


DoD  Personnel  (Cont) 


Establish  DoD-wide  software  program  management  education  and  training 

initiative 

-  Change  DSMC  and  IRMC  courses  for  PMs  to  reflect  best  commercial  practices  and  other 
recommendations  of  this  Task  Force  and  Provide  for  changes  to  reflect  the  dynamics  of  the 
software  industry 

-  Develop  and  provide  interactive  training  tools  for  senior  managers  to  perfect  software 
management  skills 

-  Rotate  government  and  contractor  personnel  between  PM  and  developer  organization  to  build 
understanding  and  trust;  encourage  use  of  IPA's  from  industry 

-  Incorporate  software  management  principles  in  senior  management  education  and  seminars 
(including  senior  services  colleges) 

-  Provide  mechanisms  for  keeping  software  expertise  current  in  the  workplace 

Develop  Acquisition  Managers  witli  software  program  management  expertise 

-  Integrate  software-qualified  personnel  into  senior  acquisition  staff 

Establish  Norms  for  the  Number  of  Software  Experts  in  Program  Offices 

Upgrade  Educational  Requirement  for  Personnel  Assigned  to  Acquisition, 

Management,  Development  and  Oversight  of  Software  Intensive  Programs 

Develop  Expertise  in  Analysis  of  Domain  Software  Design 

-  Promote  Software  Reuse  in  the  Design 


The  Task  Force  makes  the  above  recommendations  with  regard  to  DoD  software  expertise  of  its 
personnel.  The  Task  Force  strongly  recommends  an  emphasis  on  increasing  the  capability  of  its  personnel 
in  modem  software  practices  and  techniques. 


3.4  USE  AND  INTEGRATION  OF  COMMERCIAL  OFF-THE-SHELF  SOFTWARE 


Use  and  Integration  of 
Commercial  Off-the-Shelf  Software 


Findings: 

•  DoD  Does  Not  Normally  Perform  COTS  Market  Analyses  Incident  to 
Requirements  Definition 

•  DoD  is  Beginning  to  Exploit  Evolutionary  Development  Approaches 

•  Tools/Methodologies  Are  Evolving  to  Facilitate  Use  of  COTS 

-  Computer  Aided  Software  Engineering  (CASE) 

-  Object  oriented  methods 

-  Reuse  repositories 

-  Integrated  product  teams 

-  Integrated  risk  management 

_ J 


DoD  does  not  routinely  perform  COTS  software  market  analyses  during  the  requirements 
definition  phase  of  an  acquisition  program.  Nor  does  it  employ  prototypes  or  simulations  of  new 
capability  in  a  way  that  could  influence  the  requirement  process.  Rather,  requirements  are  evolved  in  a 
manner  that  is  disconnected  from  the  availability  of  commercial  applications  and  are  not  typically 
influenced  by  such  available  capability.  This  situation  is  true  for  both  hardware  and  software.  Today’s 
technology  can  facilitate  early  interaction  between  users  and  available  capability,  particularly  in  software. 
Further,  DoD  is  beginning  to  exploit  evolutionary  software  development  approaches  that  could  provide  for 
the  inclusion  of  existing  commercial  software  functionality  into  early  prototypes  and  allow  users  to  test 

Slich  CanahiliticQ  software  (tnH  mAthnHoInniAc  F»avr/*  tKpt  iic<i  rtf  PflTC 

av»v<j  VUjynUinliWO.  iTlUCtCoi  kivtkft  kV/'-'atS  MUM  lUVMIVWVtV^iVd  A1U  y  V  VI  VlVwU  UIUI  AUVimUW'  uov  v/l  v .  1  u , 

such  as  those  listed  above. 
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Use  and  Integration  of 
Commercial  Off-the-Shelf  Software 

Findings  (Cont.) 

•  DoD  has  not  fully  identified  the  pros  and  cons  associated  with  the  use  of  COTS 

-  Pros 

-  Sjvii  money  In  detign  end  development  of  thoie  components 
■  Cen  Significantly  Reduce  Development  Risk 

•  Support  !•  Concern 

•»  Selection  of  COTS  products  tarty  in  project  cycle  will  enable  requirements  to  be  driven  by  commercial  software 
capabilities 

»  Very  useful  in  rapid  prototyping 

-  £gag. 

»  Commercial  products  must  still  be  integrated  into  system  and  qualified  with  system 
*  Difficulty  in  Configuremcnt  Management  and  Support  for  Older  Releases 
»•  Additional  testing  may  be  required  to  qualify  system  with  commercial  components 
»  Subsequent  releases  determined  by  vendor,  not  DoD 

»  Security  Aspects  of  Use  of  COTS  Not  Well  Understood 

•  DoD  not  addressing  multiple  aspects 

-  D*v«lopmtnl  •nvlrown*nl 

-  Tactical  computer  prof  run 
Virwt  protxtior 
C«mm«r<Ui  Eipiunag* 

•  Claaaifitd  software  system*  a  problem  for  commercial  companies 

•  In  addition,  DoD  has  not  determined  when  to  use  COTS 

-  Commercial  Off-the-Shelf  Software  Product*  May  Not  Apply  to  AH  DoD  Systems 

-  Should  be  used  as  is  -  avoid  tailoring  or  special  features 

-  Most  weapon  cystem/real-time  application  software  for  DoD  will  not  exclusively  be  "custom,"  but 
will  involve  tome  COTS 


In  general,  DoD  has  not  identified  the  pros  and  cons  associated  with  the  use  of  COTS.  DoD  must 
learn  how  to  balance  the  cost  savings  associated  with  design  and  development  of  commercial  software 
products  and  the  significantly  reduced  development  risk  with  the  concern  for  longer  term  support  and 
system  security.  Commercial  products  must  still  be  integrated  into  and  qualified  within  each  defense 
system  and  there  is  difficulty  in  configuration  management  and  support  for  older  releases.  The  latter  point 
is  important  since  many  DoD  software  systems  are  not  deployed  before  a  commercial  software  product  is 
retired  or  replaced.  DoD  is  not  addressing  many  aspects  of  the  use  of  COTS  software: 

-  Development  environments 

-  Virus  protection 

-  Commercial  espionage 

In  essence,  DoD  has  not  developed  a  corporate  way  to  decide  when  to  use  COTS  software. 
Commercial  off-the-shelf  software  products  do  not  apply  to  all  DoD  systems.  Most  weapon  system/real¬ 
time  application  software  for  DoD  will  not  exclusively  be  "off-the-shelf."  Integration  and  configuration 
control  for  COTS  software  then  become  important  concerns.  Further,  DoD  must  learn  to  use  COTS  so 
that  it  avoids  tailoring  or  special  features. 
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Use  and  Integration  of 
Commercial  Off-the-Shelf  Software 


Recommendations 

o  Require  Trade  Studies  and  Analysis  of  the  Use  of  COTS  in  DoD's  Software  Acquisition 
Process  Where  Effective 

-  Performed  by  Acquiring  Organisation  ai  Essential  Pari  of  Defining  Requirement*  and  in  Rapid  Prototyping 
Situations 

»  Coipioy  Broad  Agency  Announcements  or  Similar  Contractual  Approaches  to  Facilitate  Such  Studies 

-  Use  of  COTS  Appropriate  When: 

•  Defining  Requirements 

»  Rapid  Prototyping  Situations 

>  Not  Required  to  Tailor  COIS  Source  Code  to  Application 
-  Not  Required  to  be  Fnor-Free 

•  COTS  Soitwase  is  *Qoee  Enough’  to  Tailor  Requirements 

•  Establish  "Customer  Friendly"  Application-Specific  Information  Technology 
"Component  Stores" 

-  Generic  Architectures  for  Specific  Domains 

-  Rapid  Requirements  Definition  Procer  an  Prototyping 
Reusable#  Pvcquallf led  Components 

-  Assemble  Systems  Rather  than  Develop  Them 

-  Reduce  Lead  Time 

-  Security  is  not  Paramount 

e  Increase  tech  base  funding  for  security  audit  tools  for  systems  employing  COTS 

•  Capitalize  on  Innovative  Cost-Effective  Techniques  for  Acquiring  and  Using  COTS 
Software  Products 

-  Such  as  Ute  of  Enterprise  Licenses 


Given  these  findings,  the  Task  Force  makes  the  above  recommendations  with  regard  to  DoD  use 
and  integration  of  commercial  off-the-shelf  software.  The  Task  Force  sees  great  benefit  to  be  gained 
through  exploitation  of  COTS  software;  however,  DoD  must  develop  corporate  approaches  to  the  use  and 
integration  of  COTS  software,  if  it  is  to  gain  this  benefit. 
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3.5  ACQUISITION 


Acquisition 


•  Findings 

-  Acquisition  practices  have  led  to: 

»  Distance  between  user  and  developer 
»  Limited  participation  by  commercial  software  companies 

-  Adherence  to  DoD  regulations  for  reviews  and  documentation  is  increasing 
software  costs 

»  DoD  software  costs  are  estimated  to  be  increased  by  at  least  20  %  over 
commercial  best  practice 

»  Commercial  best  practice  requires  much  less  documentation  than  DoD 

-  Funding  for  maintenance  planning/execution  starts  late 
»  Maintenance  assumed  organic;  inhibits  teaming/partnerships 

-  Acquisition  focus  is  on  mandatory  "how  to"  specifications  and  standards 
rather  than  the  product  (what) 

»  l  engthens  process  and  adds  costs 
»  Discourages  harmonization  with  commercial  practice 
»  Creates  adversarial  relationship 

-  Acquisition  process  does  not  reward  development  of  reusable  software 


The  Task  Force  was  concerned  that  the  specific  acquisition  and  contracting  approaches  used  by 
DoD  inhibit  use  of  commercial  practices  and  software.  The  Task  Force  findings  in  this  area  are  shown 
above.  Strict  adherence  to  DoD  regulations  for  reviews  and  documentation  is  increasing  software  costs. 
DoD  software  costs  are  estimated  to  be  increased  by  at  least  20%  over  commercial  best  practices.  The 
DoD  focus  on  detailed  technical  specifications  has  lengthened  the  process,  added  costs,  discouraged 
harmonization  with  commercial  practice,  and  created  a  highly  adversarial  relationship  between  the 
Government  and  industry.  There  is  a  strong  belief  that  certain  commercial  companies  or  divisions  of 
companies  have  opted  to  stay  away  from  government  contracts  due  to  the  complexity  of  the  acquisition 
rules  and  regulations. 
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Recommendations-Acquisition 


•  Recommendations 


-  Implement  the  following  acquisition  approach: 

»  Establish  acquisition  focus  on  functionality  and  consistency  with 
"commercial  best  practice" 

»  Revise  procedures  encouraging  interaction  between  user  and  developer 
and  achieving  early  functionality 

»  Minimize  DoD  regulations  for  review  and  documentation  that  are 
different  than  "commercial  best  practice" 

»  Require  planning  for  maintenance  at  beginning  of  development  process 

»  Provide  government  funded  vehicle  in  contracts  to  inccntivize 
development  of  reusable  SW 

-  Review  all  existing  military  standards  and  military  specifications 
pertaining  to  software  development  and  documentation,  for 
continued  applicability,  such  as  DoD-STD  2167 


To  this  end,  the  Task  Force  makes  the  above  recommendations  with  regard  to  DoD  software 
acquisition  practices.  In  essence,  these  recommendations  can  move  DoD  toward  an  acquisition  approach 
that  is  more  consistent  with  commercial  approaches,  while  not  requiring  changes  in  statutes. 
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To  implement  this  recommendation,  the  Task  Force  proposed  that  DoD  evolve  a  more  appropriate 
software  life  cycle  approach  as  depicted  above.  This  approach  provides  for  early  capability  (even  in  the 
initial  bidding  process)  and  for  a  gradual  enhancement  in  capability  over  time. 
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Acquisition  Approach 


Developing  the  RFP 


Integrated  Top  Level  Specification 

•  Performance 

•  Environment 

•  Test/Validation 


Selection 

Acquiring  Agency 

—  Evaluate  T* *chi\£cal  Solution  and  Management  Plan 

—  Evaluate  SW  Fiocets  Maturity  and  Paat  Perfonnance 
Users 

—  Define  Requirements 
Evaluate  Demonstrations 
Heavy  Participation  in  Selection 


The  Task  Force  proposed  approach  to  competition  is  shown  in  the  above  figure.  The  core 
government  role  in  such  an  approach  would  be  to: 

•  Develop  the  RFP  (no  “How  To”  statement  of  work;  no  management  CDRLs;  no  government  in- 
process  approvals) 

•  Provide  an  integrated  “Top  Level”  specification  (architecture,  COTS/reuse,  software  engineering 
environment  and  test/validation  approach). 

•  Employ  software  metrics  as  a  key  determinant, 

•  Evaluate  proposed  technical  solutions  and  the  proposed  management  plan 
The  contractor  would  then: 

•  Provide  an  execution  plan,  management  controls  and  progress  miiestones/metncs 

•  Describe  an  in-place,  mature  software  development  organization  and  relevant  domain  experience 

•  Provide  a  skills  matrix  describing  personnel  to  be  employed 

•  Identity  a  robust  development  environment  and  describe  applicable  prior  experience 

•  Describe  automated  process  control  software 

•  Describe  the  extent  to  which  peer  inspections  will  be  used 

•  Provide  a  metrics  usage  plan  and  purposes  for  which  they  will  be  used 

•  Provide  specific  reuse  and  program  management  plans 

•  Propose  specific  architecture(s)  in  executable  code 
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3.6  SOFTWARE  ARCHITECTURE 


Software  System  Architecture: 
The  Missing  Link? 


What  is  software  architecture? 

•  Software  architecture  consists  of: 

-  Software  system  components 

-  The  relationships  among  those  components 

-  Rules  for  their  composition  (constraints) 

«  Defining  document  would  address: 

-  System  functionality 

-  Software  system  components 

-  Interfaces,  standards,  protocols 

-  Execution  model 

-  Data  flows 

-  Control  flow 

-  Critical  timing/throughput  aspects 

-  Error  handling 


r 


Software  architecture  was  emphasized  by  the  Task  Force  as  a  means  for  achieving  important  ends. 
Software  architecture  consists  of  software  system  components,  the  relationships  among  these  components 
and  the  rules  for  their  composition  (constraints).  To  use  architecture  as  a  management  tool,  DoD  needs  to 
define:  system  functionality  along  with  software  system  components  to  be  employed;  interfaces, 
standards,  and  protocols  to  be  employed;  and  the  execution  model  to  include  data  flows,  control  flow, 
critical  timing/throughput  aspects,  and  error  handling  approach. 


Software  System  Architecture  (Cont.) 


Findings 

•  Why  is  it  important? 

-  Essential  for  effective  management  over  the  lifecycle 

»  Software  system  lifecycle  costs  ~  65%  for  maintenance 
»  ~  65%  of  maintenance  costs  are  due  to  changes/modifications/upgrades 

-  Software  architecture  is  a  prime  enabler  of  flexibility  and  reuse 

-  Well-formulated  architecture  might  reduce  costs  of  changes/upgrades  by  30- 
50%  ($4-$7B/year  assuming  software  expense  ~  $30B/year) 


•  Why  doesn't  it  play  a  larger  role? 

-  Focus  is  usually  on  initial  cost,  schedule,  functionality  -  not  lifecycle 

-  2167A  reinforces  this  approach  -  requires  proof  that  design  satisfies 
functionality 

-  Difficult  to  specify,  test,  etc. 

-  Not  well  understood 

V  ■  _ _ 


Software  architecture  is  a  prime  enabler  of  flexibility  and  reuse,  and  a  well-formuiated  architecture 
might  reduce  costs  of  changes/upgrades  by  30-50%  ($4-$7B/year  assuming  software  expense 
~$30B/year).  Software  architecture  has  not  been  emphasized  because  PM  focus  has  usually  been  on  initial 
cost,  schedule,  and  functionality  and  not  on  the  life  cycle.  DoD-STD-2167A  reinforces  this  approach  by 
requiring  only  proof  that  a  design  satisfies  the  required  functionality. 
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Software  System  Architecture  (Cont.) 


Findings  (Cont.) 

•  Little  emphasis  on  architecture  in  DoD  software  programs  or  regulations 

•  Impact  of  software  architecture  issue 

-  DoD  not  benefiting  from  architecture  as  a  key  tool  for: 

»  Evolutionary  development 

»  Early  (and  often)  involvement  of  users  with  functional  capability 
»  Ability  to  include  changing  commercial  technology 
»  Reuse 

»  Facilitating  requirements  change  with  minimum  cost  and  schedule 
»  Facilitating  product  line  management 

•  Insufficient  Progress  in: 

-  Developing  models/standards  for  domain  specific  software  architectures 

-  Open  system  architectures  work 

L__ _ / 


There  is  currently  little  emphasis  on  architecture  in  DoD  directives  or  regulations.  As  a  result,  DoD 
is  not  benefiting  from  architecture  as  a  management  tool.  Further,  the  Task  Force  sees  insufficient 
progress  in  developing  models  and  standards  for  domain  specific  software  architectures. 


Software  System  Architecture  (Cont) 


Recommendations 

o  Emphasize  Use  of  Software  Architecture 

-  Establish  model  and  context  for  architecture  selection 

»  Standards-based  with  emphasis  on  "implementable" 

»  Require  vendors  to  propose,  manage  and  control  the  architecture 

-  Require  delivery  of  software  architecture  definition  as  first  step  in  any 
software  acquisition 

-  Foster  migration  strategies  at  architecture  level 

•  Assign  responsibility  within  Government  for  domain  analysis  and  product 
line  developments 

•  Provide  expertise  and  resources  to  ensure  coordinated  DoD  participation 
in  commercial/international  standards  bodies  and  users  groups 


The  Task  Force  makes  the  above  recommendations  in  order  to  facilitate  greater  use  of  software 
architecture  as  a  management  tool  for  DoD  software  programs  and  activities.  In  particular,  the  Task  Force 
sees  great  value  in  requiring  the  delivery  of  software  architecture  definition  as  a  first  step  in  any  software 
acquisition.  Where  possible,  such  software  architecture  definition  should  be  operational  (i.e.,  executable). 
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3.7  SOFTWARE  TECHNOLOGY  BASE 


Software  Technology  Base 


Findings 

•  The  current  DoD  software  technology  base  does  not 
adequately  take  advantage  of  commercial  R&D 

•  Software  technology  transfer  ( internal  and  external)  is  not 
receiving  adequate  emphasis  within  DoD 

•  There  is  a  paucity  of  data  to  support  prediction  of  cost, 
schedule  and  performance 


V _ J 


The  software  technology  base  (both  defense  and  commercial)  provides  DoD  with  ample 
opportunity  for  significantly  improving  the  defense  software  acquisition  life-cycle. 
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Software  Technology  Base  (Coni.) 


Recommendations 


Provide  for  the  evolution  of  the  DoD  Software  Technology  Strategy  to  alig  > 
with  emerging  commercial  technology  and  practices 
Emphasize  Technology  Transfer  (External  and  Internal) 

-  Fund  technology  transfer  programs  in  such  topics  as: 

»  Architectural  principles, 

»  Architecture  description  languages, 

»  Standard  interfaces,  and 
»  Integration  technologies 

-  Initiate  demonstration  programs  (e.g.,  ATDs)  to  facilitate  software  technology 
insertion  into  systems.  Examples  of  candidate  criteria: 

»  Open  standards 
»  Use  of  COTS  and  GOTS 
»  Frequent  releases  to  include  a  number  of  users 
»  Multiple  platforms 

»  Satisfies  commercial  standards  and  interoperability  standards  across  DoD 
organizations 

Initiate  formal  data  collection  and  analysis 


0 


The  Task  Force  makes  the  above  recommendations  with  regard  to  DoD  software  technology  base 
investments.  The  Task  Force  supports  a  DoD  technology  base  program  that  is  more  closely  aligned  with 
the  wide  range  of  similar  efforts  ongoing  in  commercial  organizations. 
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4.0 


SUMMARY 


4.1  “ATTA  PERSONS” 


"Atta  Persons' ' 


•  Electronic  Systems  Command  Software  by  Assembly  of  Modules 

-  Software  "Component  Store" 

•  A-109  Acquisition  Methodology 

•  Use  of  Commercial  Practices  and  Products  in  Reserve 
Component  Automation  System  (RCAS  ) 

•  Software  Reuse,  Prototypes  and  User  Involvement  of  Global 
Command  and  Control  System  (GCCS)  Initiative 

•  Perry-Paige  Initiative  Toward  Migration  Systems 


V _ J 

The  Task  Force  identified  several  ongoing  efforts  worthy  of  note. 

•  The  Air  Force  Electronic  Systems  Command  (ESC)  has  defined  an  approach  for  software  system 
development  through  the  assembly  of  pre-qualificd  software  modules  (PRISM).  ESC  is  pursuing 
the  development  of  such  software  modules.  The  Task  Force  was  veiy  supportive  of  this  program. 

•  Office  of  Management  and  Budget  (OMB)  Circular  A-109  outlines  an  approach  for  major  Federal 
system  acquisitions  that  encourages:  definition  of  top  level  needs  vs.  detailed  specifications; 
exploitation  of  innovative  private  sector  contributions  and  use  of  early  competitive  demonstrations 
of  competing  approaches.  These  all  are  acquisition  attributes  recommended  by  the  Task  Force. 

•  The  Reserve  Component  Automation  System  (RCAS)  is  an  ongoing  MIS  development  to  support 
the  reserves.  The  program  has  successfully  employed  the  A-109  acquisition  approach  and 
extensively  used  commercial  acquisition  practices  and  products.  The  Task  Force  commends  this 
approach. 

•  The  Global  Command  and  Control  System  (GCCS)  is  an  initiative  of  the  Joint  Staff  (J3  and  J6)  to 
provide  vertical  and  horizontal  interoperability  of  combat  information  systems  across  Services, 
Combatant  Commands  and  Agencies.  It  was  highlighted  for  its  software  reuse  approach,  its  use  of 
prototypes  and  its  emphasis  on  operational  user  involvement. 

•  The  Perry-Paige  migration  systems  initiative  has  established  a  focus  on  selecting  a  set  of  target 
computing  systems  (including  MIS,  C3I  and  embedded)  towards  which  DoD  will  aim.  This 
migration  strategy  will  enable  a  more  cost-effective  DoD  investment  in  software  across  the  life- 
cycle. 
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4.2  OVERARCHING  RECOMMENDATION 


Overarching  Recommendation 


•  SECDEF  Assign  USD  (A&T)  the  Responsibility  for  DoD-Wide  Software 
Technology,  Policy,  Practices  and  Acquisition 

•  This  Responsibility  Includes  Allocating  Resources  and  Assigning 
Responsibility  for: 

-  Drafting  and  Institutionalizing  a  New  Software  Acquisition  Policy  That 
Includes  the  Recommendations  of  this  DSB  Task  Force 

-  Creating  Incentives  and  Ensuring  Compliance  to  the  Policy 

-  Creating  Terms  and  Conditions  and  Financial  Rewards  to  Maximize  the 
Ability  of  DoD  to  Realize  the  Benefits  of  Commercial  Best  Practices 

-  Requiring  Source  Selection  Evaluation  of  Development  Contractors  Through 
a  Formal  Software  Process  Capability  Evaluation 

-  Advising  the  Acquisition  Executive  on  Matters  Concerning  Software 
Technology,  Acquisition  and  Architecture  for  Major  Programs 

-  Maintaining  a  Digest  of  Lessons  Learned  and  Best  Practices  and 
Communicating  the  Same  to  Program  Managers  and  Contractors 


Throughout  its  deliberations,  the  Task  Force  acknowledged  that  the  issues  associated  with  defense 
software  were  applicable  across  the  spectrum  of  DoD  software  intensive  systems.  The  Task  Force  also 
frequently  learned  of  obstacles  based,  in  part,  on  the  current  dichotomous  DoD  structure  associated  with 
software  technology,  policy,  practices  and  acquisition.  In  order  to  ensure  that  the  DoD  reap  maximum 
benefit  from  its  recommendations,  the  Task  Force  formulated  the  above  overarching  recommendation. 
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Overarching  Recommendation  (Cont.) 


•  In  Carrying  Out  This  Responsibility,  Consider  Forming  an  Executive 
Council 

-  Including  the  DDR&E,  the  ASD  (C3I)  and  Appropriate  Representatives  from 
the  Services  and  Defense  Agencies 

-  Provide  Supporting  "Process  Action  Team"  to  Assist  in  Implementation 


The  Task  Force  also  identified  the  mechanism  by  which  its  overarching  recommendation  could  be 
readily  implemented. 


APPENDIX  A 
TERMS  OF  REFERENCE 


THE  UNDER  SECRETARY  OF  DEFENSE 


ACQUISITION 


WASHINGTON,  DC  20301-3000 


M.  0  6'  1993 


MEMORANDUM  FOR  CHAIRMAN,  DEFENSE  SCIENCE  BOARD 

SUBJECT:  Terms  of  Reference — Defense  Science  Board  Task  Force  on 

Acquiring  Defense  Software  Commercially 


You  are  requested  to  form  a  Defense  Science  Board  Task  Force 
on  Acquiring  Defense  Software  Commercially.  Determine  the 
conditions  under  which  the  procurement  of  defense  software  (i.e„, 
operational  software,  support  software,  and  software  tools)  can 
appropriately  use  commercial  practices  and  develop  a  strategy  for 
defense  software  procurement  that  substantially  incorporates  such 
practices.  Specifically  address  within  this  strategy  DoD  use  of 
commercial  software  products. 

The  scope  of  this  effort  should  include  all  DoD  systems  that 
are  software  intensive.  It  should  address  all  stages  in  the  life 
cycle  of  a  software  component  from  initial  procurement  to 
evolutionary  upgrade  of  software  or  of  software/hardware 
combinations.  This  commercially-based  strategy  should  not  be 
constrained  by  existing  DoD  standards;  it  should  be  viewed  as  a 
coexisting  alternative  to,  rather  than  replacement  of,  the 
current  DoD  procurement  strategy.  Accordingly,  you  should 
compare  your  proposed  strategy  to  the  current  DoD  strategy  to 
indicate  circumstances  in  which  each  strategy  is  most  beneficial. 

To  assess  the  appropriateness  of  DoD  use  of  commercial 
practices,  the  Task  Force  should  identify  and  apply  objective 
measures  such  as  elapsed  development  time,  software  life-cycle 
cost,  management  risk,  as  well  as  measures  of  software  product 
quality. 

The  Task  Force  should  consider  at  least  the  following 
topics : 


•  Technical:  state-of-the-art  and  best  commercial 
practices,  development  tools,  reusable  software  components, 
and  techniques  and  tools  for  tailoring  commercial  software 
components  for  use  in  defense  systems. 

•  Management:  DoD  management  of  the  commercial  development 
process,  software  risk  management  techniques  and  supporting 
tools,  minimum  delivery  time,  affordability,  maintenance 
after  product  delivery,  post-deployment  product  enhancement, 
software  process  support  tools,  quality  and  assured 
availability,  and  use  of  development  and  maintenance  tools. 


•  Contracting:  technical  data  rights,  intellectual 
property  rights,  liability,  alternative  forms  of  procurement 
agreements,  and  incentives  for  creation  of  reusable  software 
components  as  well  as  their  subsequent  reuse. 

The  Director,  Defense  Research  and  Engineering  will  sponsor 
this  study.  Dr.  George  H.  Heilmeier  and  Dr.  Larry  E.  Druffel 
will  serve  as  Co-Chairmen.  The  Office  of  the  Director,  Defense 
Research  and  Engineering  will  provide  the  necessary  funding  and 
support  contractor  arrangements.  The  Executive  Secretary  will  be 
Ms.  Virginia  L.  Castor.  Commander  Robert  C.  Hardee  will  be  the 
Defense  Science  Board  Secretariat  representative.  It  is  not 
anticipated  that  this  Task  Force  will  need  to  go  into  any 
"particular  matters"  within  the  meaning  of  Section  208  of  Title 
18,  U.S.  Code,  nor  will  it  cause  any  member  to  be  placed  in  the 
position  of  acting  as  a  procurement  official. 


APPENDIX  B 
BRIEFINGS 

PROVIDED  TO  THE  TASK  FORCE 


Briefings  Provided  to  the  Task  Force 


Joint  Staff  Views  on  How  DoD  Can  Adopt  Commercial  Software 
Practices 

MG  David  Kelley,  Vice  Director  J6 

Army  Views  on  How  DoD  Can  Adopt  Commercial  Software  Practice 

LTG  Peter  Kind,  Director,  Information  Systems  for 

Naval  Views  on  How  DoD  Can  Adopt  Commercial  Software  Practice 

RADM  John  Hekman,  Commander,  NISMC 

Air  Force  Views  on  How  DoD  Can  Adopt  Commercial  Software 

Practices 

Mr.  Lloyd  Moscmann,  Deputy  Assistant  Secretary  of  Air 

Force  (Communication,  Computers  &  Support  Systems) 

DISA  Views  on  How  DoD  Can  Adopt  Commercial  Software  Practices 

Dr.  Mark  Scher,  Director  of  Infrastructures,  Defense 

Information  Systems  Agency 

DoD  5000  Series  Regulations 

Mr.  Gene  Porter,  Director,  Acquisition  Program  Integration 

DoD  8000  Series  Regulations 

Mr.  Harry  Pontius,  Director,  C^l  Policy 

DoD-STD-2 1 67A/MIL-STD-498 

Dr.  Singh,  Senior  Manager,  Space  &  Naval  Warfare  Systems 
Command 

Draft  White  Paper  on  Software  Acquisition 

Dr.  Jack  Ferguson,  Program  Manager,  Software  Engineering 
Institute 

Data  Modeling  vs.  Data  Standards 

LTG  Peter  Kind,  Director,  Information  Systems  for 

Case  Study:  B2 

Mr.  Fred  Schwartz.  Director  of  Engineering, 

Case  Study:  Ensemble  of  Real-time  Software  Systems 

MajGen  Israel,  Director,  Defense  Airborne  Reconnaissance. 
Office 

Case  Study:  Reserve  Component  Automation  System  (RCAS) 

MG  Gary  Stemley,  Program  Manager  for  RCAS 

Case  Study:  AEGIS 

CAPT.  Richard  Cassidy,  AEGIS  Technical  Director 

Case  Study:  PRISM 

Mr.  Robert  Kent,  ESC/ENS 

Case  Study:  BMC3 

Lt  Co!  Robert  Phelps,  BMDO 

Boeing  Views  on  DoD  vs.  Commercial  Software  Practices 

Mr.  John  Hanson,  Director,  Systems  Software  &  Engineering, 
Boeing 

McDonnell  Douglas  Views  on  DoD  vs.  Commercial  Software  Practices 

Mr.  L.  George  Hite,  Senior  Manager,  McDonnell  Douglas 

IBM  Views  on  DoD  vs.  Commercial  Software  Practices 

Mr.  Dave  Coyer,  Program  Manager,  IBM  Federal  Systems 
Company 

Object  Technology  and  a  COTS  Product  Called  "SNAP” 

Mr.  Joe  Fox,  Chairman,  Template  Software  Inc. 

Procurement  Regulations/Issues 

Mrs.  Eleanor  Spector,  Director,  Defense  Procurement 
OUSD(A&T)  i 

Integrated  Computer  Aided  Software  Engineering  (ICASE)  Program 

Col  Gary  Case,  ICASE  System  Program  Director 

Computer  Sciences  Corporation  Case  Study  on  Commercial  Software 
Development 

Mr.  Daniel  Kcntp,  Senior  Partner.  Computer  Sciences  Corp. 

Commercial  Software  Acquisition  Practices 

Mr.  Alex  Morrow,  General  Manager,  Lotus  Development 

Corp. 

Experiences  in  DoD  Software  Maintenance 

Mr.  Bobby  McDonald,  Deputy  Director,  Electronic  Warfare 
Directorate,  Warner  Robins  ALC 

MITRE  Corporation  Experiences  in  Exploiting  Commercial  Practices 

Mr.  Sieven  Crisp,  Dept.  Head,  MITRE  Corp. 

FAA  Study  on  the  Use  of  Commercial  Software 

Mr.  John  Stenbit,  Vice  President  &  General  Manager.  TRW 
Systems  Integration  Group 

NSIA  Study  on  the  Use  of  COTS  Software 

Gen.  William  Richardson,  USA(Ret),  Executive  Vice  President 
Army  Programs,  Burdeshaw  Associates,  Ltd.  and  Ms.  Linda 
Connor,  Program  Manager,  Rockwell  International 

DoD  Software  Reuse  Initiative 

Ms.  Linda  Brown,  ODASD(lM)/Information  Technology 

MITRE  Study  on  Feasibility  of  Using  a  Software  Acquisition  Maturity 
Model  in  DoD 

Ms.  Judith  Clapp,  Division  Assitant,  The  MITRE  Corp. 

Software  Management  Metrics  and  Reliability 

Mr.  Douglas  Putnam  and  Mr.  Lawrence  Putnam,  Jr., 

Quantitative  Software  Management,  Inc. 

Innovative  Techniques  in  DoD  SW  Acquisition:  F-22  Integrated 
“roduct  Team  Approach 

Colonel  Robert  Kayuha 

I  Legal  Impediments  to  Acquiring  Defense  Software  Commercially 

Mr.  Robert  Gorman,  OSD  General  Counsel 
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APPENDIX  C 

COMPARISON  OF  SOFTWARE 
ACQUISITION  METHODS 


Comparison  of  Software  Acquisition  Methods1 


Requiremen 

ts  Definition 

Best  Commercial  Practice 

Current  DoD  Practice 

Requirements  based  on  strategic  plan  and  market 
analysis. 

Requirements  based  on  using  command  Mission 

Need  Statement,  master  plans,  and  top-level 
certification. 

Requirements  based  on  life-cycle  resource 
constraints. 

Requirements  based  largely  on  annual  budget 
resource  constraints. 

Detailed  requirements  generated  by  interdisciplinary 
team  including  users,  domain  experts,  and  system 
engineers. 

Detailed  requirements  generated  by  buyer  in 
collaboration  with  user.  Team  generally  includes 
domain  experts  and  acquisition  personnel. 

Buyer,  user,  and  vendor  are  a  team.  Attitude  cf 
partnership,  trust  and  cooperation.  Presumption  of 
trustworthiness  for  reputable  commercial 
organizations. 

"Us  vs.  them"  mentality  about  contractors. 

Government  thinks  in  terms  of  control, 
accountability,  detailed  auditing,  and  double 
checking.  Presumption  that  contractors  cannot  be 
trusted. 

Functional  specification  is  modified  by  knowledge 
of  availability  of  existing  products. 

Functional  and/or  performance  specification;  little  to 
no  regard  for  existing  products. 

Vendors  involved  early  in  study,  analysis  and 
prototyping  with  emphasis  on  reuse  and  evolution 
of  existing  systems. 

May  contract  for  prototypes,  but  contractor 
involvement  in  pre-award  discussions  is 
discouraged. 

Need  is  based  on  business  case  and  decisions  are 
based  on  return  on  investment. 

("time  to  make") 

Need  based  on  Mission  Need  Statement;  decisions 
based  on  need,  economics,  and  politics,  ("time  to 
field") 

Efficient  decision  processes. 

Decision  processes  formal  and  time-consuming. 

Level  of  documentation  is  negotiable  based  on 
individual  user  needs  and  complexity  of  system 
being  developed. 

Extensive  (often  redundant  or  unnecessary) 
documentation  required  under  2167 A. 

Tailoring  of  documentation  requirements  is  often 
minimal  or  discouraged. 

More  detailed  analysis  of  cost  versus  feature. 
Dropping  lower  value/higher  cost  options  or 
reducing  requirements  is  practiced. 

Little  or  no  requirements  reductions  on  high  cost 
items. 

More  requirements  trade-off  decisions  (involving 
complexity  and  schedule)  for  reduced  time  to  field. 

Very  little  flexibility  to  trade-off  requirements  creep 
versus  complexity  and  schedule. 

Selected  vendors  may  assist  in  preparing 
specifications. 

Vendors  not  involved  in  preparing  specifications. 

Tools  used  to  create  system  models  for  use  in 
requirements  definition;  e.g.,  GUI  Building. 

Requirements  definition  based  on  Mission  Need 
Statement. 

Flexibility'  allowed  in  choice  of  programming 
language. 

Evolutionary  and  incremental  approach  favored. 

Requirements  defined  up  front  with  little  flexibility 
for  modifications. 

Summary 

Commercial  is  more  flexible  and  open  between  users  and  suppliers,  and  requirements  are  based  on  a 
strategic  plan.  In  the  commercial  world,  there  is  more  willingness  to  adjust  requirements  based  on 
availability  of  products  and  thus  to  filed  a  system  sooner  and  evolve  it  to  include  more  capability  at 
significant  cost  savings. 

1  This  appendix  was  initially  derived  from  a  White  Paper  on  software  acquisition  methods  prepared  by  the  Software 
Engineering  Institute.  The  resulting  content  represents  the  consensus  of  this  Task  Force. 
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Vendor  Selection 


Best  Commercial  Practice 

Current  DoD  Practice 

Solicit  multiple  (but  not  all)  qualified  vendors  -a 
selected  few.  Encourage  teaming  with  a  view  to 
attaining  a  long  term  relationship  that  covers  the 
entire  life  cycle  and  fosters  trade-offs  in  cost  and 
schedule. 

Solicit  all  possible  vendors.  Vendor  proposals  must 
meet  100%  of  requirements.  Teaming  seldom 
encouraged;  development  and  maintenance  usually 
separ  ate  entities. 

Compare  vendor  history  and  experience.  Maintain 
long-term  relationships. 

Can  compare  previous  performance,  but  normally 
can't  have  long-term  relationships. 

The  organization  that  will  be  responsible  for  a 
system  over  its  full  life  cycle  is  heavily  involved 
from  the  beginning. 

Maintenance  organization  not  usually  involved  in 
vendor  selection  process. 

Use  site  visits  and  demonstrations  to  gain 
knowledge  of  vendor  capabilities. 

Site  visit  only  by  capability  evaluation  team,  or 
other  expert  teams.  Visits  are  very  structured. 

Negotiate  for  best  values  based  on:  (1)  trade-offs  of 
costs  and  requirements  licensing;  and  (2) 
consideration  of  both  vendor  and  buyer  best 
interests. 

Negotiation  based  on  lowest  cost  and  shortest 
schedules,  endangering  maturity  of  finished 
product. 

Overall  goals:  (1)  obtain  product  at  reasonable  cost 
as  soon  as  possible;  and  (2)  achieve  the  business  case 
for  the  system. 

Overall  goal:  Obtain  lowest  cost  product  that 
rigorously  meets  all  requirements,  but  be  fair. 

Relatively  few  review  and  approval  steps  once 
vendor  is  selected. 

Review  and  approval  process  more  structured  and 
complex  once  vendor  selected. 

Past  performance  considered,  but  only  as  a  minor 
factor. 

More  flexibility  in  vendor  selection  based  on  metrics 
and  overall  assessment. 

Selection  of  vendor  forced  by  use  of  pre-defined 
metrics  for  proposal  evaluation. 

Modifications  made  as  procurement  proceeds  in 
order  to  get  best  results. 

Change  difficult  once  process  begins. 

Summary 

Very  different  processes  with  commercial  much  more  flexible,  but  with  no  requirement  for  fairness,  or  to 
maintain  the  public  trust.  Commercial  encourages  vendors  to  offer  best  solution,  but  solution  may  not 


meet  100%  of  the  requirements.  Teaming  and  long-term  relationships  are  more  easily  accommodated  by 
industry. _ 


|  Development  Process  | 

Best  Commercial  Practice 

Current  DoD  Practice 

Vendor  often  tailors  existing  systems  and  uses 
COTS.  System  designed  to  fit  in  a  defined  product 
or  product  line  architecture. 

Varies  with  application.  Some  systems  use  COTS. 
However,  usually  a  new  system  that  doesn't  reuse 
legacy  software.  Unique  systems  are  built  with  little 
regard  for  architecture. 

Buyer  may  have  heavy  involvement  in  design  and 
development  as  part  of  the  team  (Integrated  Product 
Development  team). 

Formal,  structured  spiral,  or  waterfall  model.  Buyer 
oversees,  but  team  approach  is  not  usually 
emphasized. 

Reviews  typically  informal  and  stress  progress 
against  goals. 

Reviews  usually  very  formal  and  include  technical 
design  details  in  addition  to  progress  metrics. 

Buyer  actively  involved  as  co-participant  in 
management  of  technical  details. 

Micro  management  of  technical  details. 

Heavy  user  involvement. 

Limited  user  involvement.  Heavy  buyer 

involvement. 

Vendor  embraces  one  or  more  industry  standards 
which  improves  interface  and  integration  with  COTS 
products. 

Government  and  industry  standards  called  out.  Not 
all  government  standards  enhanced  by  COTS 
products. 

Buyer  requirements  may  be  translated  to  more 
"general  purpose"  requirement  for  potential 
software  reuse. 

Tailored  system;  little,  if  any,  focus  on  designing  in 
reusable  code. 

Management  reviews  and  degree  of  oversight  are 
commensurate  with  size  and  risk  of  program. 

Notably  more  detailed  reviews  and  oversight 
performed. 

Prototyping  common,  with  joint  applications 
development  teams  (user  and  developer)  working  to 
clarify  requirements  and  incorporate  new 
requirements  that  do  not  affect  cost  or  schedule. 

Prototyping  seldom  used. 

Use  of  flexible  architectures  allows  insertion  and 
plug  and  play  of  COTS  products. 

Use  of  MIL  standard  computers  and  legacy 
instruction  set  architectures  restricts  new 
development. 

Summary 

Commercial  more  flexible  with  likelihood  of  a  team  approach  and  is  biases  toward  reuse  and  tailoring  of 
existing  systems.  Multi-year  acquisitions  not  re-justified  each  year.  Product  improvements  are  anticipated. 
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Business  Practices 


Best  Commercial  Practice 


Informal  contracting,  joint  ventures,  partnerships 
with  mutual  economic  benefit  and  vested  interest  in 
success. 


Oversight  built  on  established  relationships. 


Current  DoD  Practice 


Difficult  to  write  contract  to  motivate  contractors  to 
reduce  cost;  expensive  to  terminate  contractors. 


Burdensome  cost  accounting  procedures  required; 
extensive  oversight,  reporting,  and  documentation 
requirements. 


Government  personnel  regulations,  policies,  and 
practices  determine  qualifications  of  its  managers, 
rotations  of  assignment,  and  training. 


Multi-year  effort.  Yearly,  unpredictable  fundin 


Summary 

Commercial  practice  more  flexible  with  greater  incentives. 


Can  hire  and  fire  vendors  and  managers. 


Multi-year  effort  and  fundin 


Integration  Testing  and  Delive 


Best  Commercial  Practice 


Unless  system  is  for  a  new  plant,  then  there  are 
major  "cut-over"  issues. 


Sometimes  difficult  to  assemble  complete  system  in 
laboratory  environment  due  to  cost. 


Current  DoD  Practice 


Similar'  "cut-over"  or  transition  issues. 


Usually  integrate  system  in  laboratory  prior  to 
operational  testing. 

Development  testing  vs.  operational  testing  via 
statutory  mandate. 


Little  beta  testin 


Structured,  specified  operational  testing  conducted 
by  separate  authority.  Acceptance  authority  rests 
with  buyer. 


Emphasis  on  DoD  as  oversight  and  approval 
authori 


Summary 

Integration  and  functional  testing  seem  appropriate  to  the  need.  DoD  use  of  separate  test  agency  adds  time 
and  complexity.  Absence  of  beta  testing  increases  cost  to  DoD. 


Beta  testing  widely  used  to  quickly  find  errors. 


Ultimate  acceptance  authority  rests  with  buyer,  not  a 
separate  organization. 


Buyer/user/vendor  are  a  team. 
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Rights  in  Data 


Best  Commercial  Practice 

Current  DoD  Practice 

if  custom  development,  buyer  gets  all  rights,  but 
vendor  may  retain  right  to  subsequent  sales. 

Specified  by  contract. 

Government  usually  demands  all  rights  for 
government  use  due  to  organic  support  and 
maintenance  needs,  and  competition  (via  statutory 
mandate). 

If  tailored  version  of  standard  system,  buyer  only 
gets  rights  to  tailored  parts. 

Same  as  commercial.  May  have  exceptions  for 
proprietary  material. 

Summary 

Similar,  but  commercial  is  a  little  more  flexible,  especially  regarding  resales. 

Maintenance  ] 

Best  Commercial  Practice 

Current  DoD  Practice 

Organic  support  shifting  to  outsourcing  or  vendor. 

Organic  support,  with  reluctance  to  be  dependent  on 
vendor.  Use  of  depot  maintenance  makes 
interoperability  issues  more  manageable.  Also, 
must  be  responsive  to  user  for  critical  systems. 

Level  of  discrepancy  reporting  required  is  based  on 
user  needs.  Problem  resolution  usually  delayed 
until  next  major  release  of  software.  Buyer  has 
limited  power  to  implement  fast  resolution  of 
problems. 

Bureaucratic  and  paper-intensive  discrepancy 
reporting  and  change  control  board  often  imposed. 
Award  fees  may  be  tied  to  problem  workoff, 
resulting  in  fast  resolution  of  problems. 

COTS  solutions  take  advantage  of  economics  of 
scale  since  maintenance  costs  are  leveraged  across 
multiple  buyers. 

Unique  software  requires  custom  maintenance  all 
borne  by  the  single  buyer. 

Summary 

The  DoD  and  industry  currently  have  different  requirements,  and  trends  are  moving  apart.  However,  DoD 
is  currently  reevaluating  its  practices  for  hardware  systems  and  perhaps  should  also  reevaluate  for  software. 
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APPENDIX  D 

PROPOSED  ACTION  PLAN  FROM 
SERVICE  SOFTWARE  EXECUTIVE 

OFFICIALS 


OHlct,  Director  oi  Information 
Syatamt  for  Command,  Control. 
Communication*.  k  Computer* 


DEPARTMENT  OF  THE  ARMY 
OFFICE  OF  THE  SECRETARY  OF  THE  ARMY 
107  ARMY  PENTAGON 
WASHINGTON  DC  20310-0107 


MEMORANDUM  FOR  THE  EXECUTIVE  SECRETARY,  DEFENSE  SCIENCE  BOARD 
TASK  FORCE  ON  DOD  ACQUISITION  OF  COMMERCIAL  SOFTWARE 

SUBJECT:  Proposed  Action  Plan  with  Regard  to  Common  Changes  Proposed  by  DoD 
Speakers  to  Revise  the  Software  Acquisition  Process 


The  DSB  Task  Force  asked  us  to  provide  a  list  of  common  recommended 
changes  needed  to  revise  DoD's  software  acquisition  process.  Our  recommended  list 
of  proposed  changes  is  provided  in  the  enclosure  to  this  memorandum.  As  we  worked 
together  to  develop  this  list,  we  found  it  useful  to  categorize  the  recommended  changes 
into  five  major  categories.  Several  of  our  recommendations  did  not  receive  adequate 
emphasis  in  our  presentations,  yet  are  important  to  the  process  we  are  studying.  We 
feel  that  if  OSD  is  serious  about  addressing  Software  Acquisition  issues,  a  single 
senior  office  with  primary  responsibility  for  effecting  the  reforms  proposed  by  the  DSB 
will  be  essential.  Further,  such  an  official  or  office  must  receive  "from  the  heart" 
sustainment  and  backing  from  the  highest  levels  within  OSD.  The  enclosed  list 
addresses  our  concerns  with  regards  to  the  questions  before  the  DSB.  We  hope  that 
this  will  be  beneficial  to  the  DSB  Task  Force  and  we  all  look  forward  to  continued  active 
participation  in  this  effort. 


PETER  A.  KIND 
Lieutenant  General,  GS 
Director  of  Information 
Systems  for  Command,  Control, 
Communications  and  Computers 


Deputy  Assistant  Secretary 
(Communications,  Computer,  and  Support 
Systems) 


Date 


4  Feb  94 


Date_  9/^6  7  j 


^ 

^oWg.  hekman 

'  Rear  Admiral,  SC,  USN 


Commander,  Naval  Information 
Systems  Center 


Date 


4  Feb  94 
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JOINT  RECOMMENDED  PROPOSED  ACTIONS 

I.  DESIGNATE  SINGLE.  COMMON  DOD  SOFTWARE  MANAGEMENT  OFFICIAL 
(ACTION  OFFICE  (USD)) 

A.  ESTABLISH  DEPUTY  ASSISTANT  SECRETARY  OF  DEFENSE 
(SOFTWARE) 

B.  PROVIDE  SUFFICIENT  RESOURCES  FOR  MISSION 

II.  SOFTWARE  ACQUISITION  AND  LIFE-CYCLE  MANAGEMENT  (ACTION  OFFICE 
NEW  DASD) 

A.  PROMOTE  REUSE  BASED  ON  DOMAIN  MANAGEMENT 

1 .  Define  domains  of  interest  or  "areas  of  business" 

2.  Assign  responsibilities  to  manage  the  domains 

3.  Establish  and  manage  common  architectures  within  the  domains 

B.  REDUCE  MONITORING  OF  MATURE  COMPANIES 

1 .  Identify  criteria  for  assessing/determine  process  maturity 

2.  Consider  maturity  in  source  selection 

3.  Allow  sole  source  extension  for  high  quality  vendors 

C.  PROMOTE  GOVERNMENT/INDUSTRY  TEAMING 

1 .  Reduce  Documentation  Requirements  to  a  minimum 

2.  Provide  for  electronic  delivery/evaluation/exchange 

D.  MANAGE  RISK 

1 .  Mandate  the  use  of  market  research 

2.  Use  metrics  effectively  for  management 

3.  Develop  Risk  Management  Disciplines 

4.  Stick  with  a  winner/punish  poor  performers 

III.  POLICY  AND  STANDARDS 

A.  ADOPT  COMMERCIAL  STANDARDS  (ACTION  OFFICE(A5D(C31))) 

1 .  DoD  invests  in  commercial  standards  developments 

2.  Change  FAR7DFAR  and  SD-1  as  appropriate 

B.  REVISE  THE  MILESTONES  FOR  SOFTWARE-INTENSIVE 
DEVELOPMENTS  (ACTION  OFFICE  USD(A&T)/ASD(C3I)) 

1 .  Address  the  need  for  a  software-first  philosophy 

2.  Provide  for  a  layered  software/hardware  standards  based  architecture 


C.  DEV  ELOP  DOD  "BETA  TEST"  PHILOSOPHY  (ACTION  OFFICE  OSD(T&E) 

1 .  Team  with  Universities/other  appropriate  activities 

2.  Allow  fielding  of  software  direct  from  test  beds  with  user  consent 

IV.  PERSONNEL 

A.  DEVELOP  SOFTWARE  ACQUISITION  MANAGERS  (ACTION  OFFICE 
USD(A&T)) 

1 .  Provide  a  career  path  for  software  managers 

2.  Integrate  software  personnel  into  senior  acquisition  staff 

B.  PROVIDE  SOFTWARE  EDUCATION  FOR  SENIOR  MANAGERS 
(ACTION  OFFICE  USD(A&T)) 

i  Develop  DoD  Senior  Software  Managers  Course 
2.  Develop  and  provide  interactive  training  tools  for 
senior  managers  to  pei  feet  software  management  skills 

C.  PUBLISH  AND  PROMOTE  "BEST  PRACTICES"  HANDBOOK  (ACTION 
OFFICE  OSD(T&E)) 


D.  ENSURE  DOMAIN  EXPERTISE  (ACTION  OFFICE  MILITARY 
DEPARTMENTS/  AGENCIES) 


V.  SOFTWARE  TECHNOLOGY  BASE  AND  TRANSITION  (ACTION  OFFICE 
DDR&E) 


A.  PROVIDE  FOR  SOFTWARE  TECHNOLOGY  INSERTION  INTO  SYSTEM 
ACQUISITION 

B.  INVEST  IN  THE  DOD  SOFTWARE  TECHNOLOGY  STRATEGY 

C.  PROVIDE  FOR  THE  EVOLUTION  OF  THE  DOD  SQFTW ARE 
TECHNOLOGY  STRATEGY  TO  CAPTURE  EMERGING  COMMERCIAL 
PRACTICES 


